Skip to content

Add a test for non-ascii char input escaped with C-v#835

Merged
tompng merged 1 commit into
ruby:masterfrom
tompng:non_ascii_escape_test
Jul 5, 2025
Merged

Add a test for non-ascii char input escaped with C-v#835
tompng merged 1 commit into
ruby:masterfrom
tompng:non_ascii_escape_test

Conversation

@tompng

@tompng tompng commented Jul 5, 2025

Copy link
Copy Markdown
Member

There was a mystery of 0x16 in Reline. There is no test and no source code comment.
0x16 seems to be inserted in some case (what case?) in some way (how?) but nobody knows how and why.

(c == 0x16 && @input.tty? && @input.raw(min: 0, time: 0, &:getbyte)) || c

It turned out that this is 0x16 came from a configuration of terminal emulator Escape non-ASCII input with Control-V
(ohmyzsh/ohmyzsh#7412)

ctrlv_escape_config

With this configuration, every non-ascii byte will be escaped with \C-v

irb> STDIN.raw{STDIN.readpartial(100)} # input あ = "\xE3\x81\x82"
=> "\x16\xE3\x16\x81\x16\x82"

# Bracketed paste also includes \x16
irb> print "\e[?2004h"; STDIN.raw{STDIN.readpartial(100)+STDIN.readpartial(100)} # paste あ
=> "\e[200~\x16\xE3\x16\x81\x16\x82\e[201~"

Now we can add a test and a comment

@tompng tompng force-pushed the non_ascii_escape_test branch 2 times, most recently from 2ec6648 to 5d6f891 Compare July 5, 2025 13:54
sync
@pty_input.write(str)
try_sync
end

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ideally, I think changing yamatanooroti's write to not do this conversion is better.
But if we do that, there will be no way to input meta key to yamatanooroti-windows and yamatanooroti-vterm in a common way.

C-v are inserted by terminal emulator's configuration.
In macOS Terminal.app, enabling "Escape non-ASCII input with Control-V" configuration
will make the input "\xE3\x81\x82" to "\x16\xE3\x16\x81\x16\x82".
@tompng tompng force-pushed the non_ascii_escape_test branch from 5d6f891 to 8fe8591 Compare July 5, 2025 14:01
@tompng tompng merged commit 148b744 into ruby:master Jul 5, 2025
44 checks passed
@tompng tompng deleted the non_ascii_escape_test branch July 5, 2025 14:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant